{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 逻辑回归解决多分类问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OvR(One vs Rest)\n",
    "> 每次计算一个特征和其他，轮流对每个特征执行这种策略，就完成了多分类问题\n",
    "![OvR](images/OvR.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OvO(One vs One)\n",
    "> OvO在sklearn中叫“multinomial\", 需要配合使用\"newton-cg\"作为solver.ovo更加准确，没有混淆其他信息"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![OvO](images/OvO.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 代码实战OvR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = datasets.load_iris()\n",
    "X = iris.data[:, :2] # 取前2列\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=666)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='ovr', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='warn', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression # sklearn中的LogisticRegression已经内置了OvR和OvO的功能\n",
    "log_reg = LogisticRegression(multi_class='ovr') # 多分类的核心参数，使用OvR分类\n",
    "log_reg.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6578947368421053"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_reg.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_decision_boundary(model, axis):\n",
    "    \"\"\"\n",
    "    根据模型绘制决策边界\n",
    "    \"\"\"\n",
    "    x0, x1 = np.meshgrid(\n",
    "        np.linspace(axis[0], axis[1], int((axis[1]-axis[0])*100)).reshape(-1, 1),\n",
    "        np.linspace(axis[2], axis[3], int((axis[3]-axis[2])*100)).reshape(-1, 1),\n",
    "    )\n",
    "    X_new = np.c_[x0.ravel(), x1.ravel()]\n",
    "\n",
    "    y_predict = model.predict(X_new)\n",
    "    zz = y_predict.reshape(x0.shape)\n",
    "\n",
    "    from matplotlib.colors import ListedColormap\n",
    "    custom_cmap = ListedColormap(['#EF9A9A','#FFF59D','#90CAF9'])\n",
    "    \n",
    "    plt.contourf(x0, x1, zz, linewidth=5, cmap=custom_cmap)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:17: UserWarning: The following kwargs were not used by contour: 'linewidth'\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df5BcdZnv8fczP8yEySQhQkhIookFwmr4EUnlolkRxLKWiIQrWHJLBSw1pbu4qOtyLxsLhJJrqbcsWXG1olwNuterBpYgwpaiQQEXqAQUAkFkb5QkTBIEksyEGczMPPeP7p70dLrPOd2nT58+fT6vqhTT5+fTZ5pnvv3tp59j7o6IiHS+rrQDEBGR1lDCFxHJCSV8EZGcUMIXEckJJXwRkZxQwhcRyYnICd/Mus3sUTO7s8q6y83seTP7bfHfR5obpoiIxNVTx7ZXAtuAmTXW/9Ddr4gfkoiIJCHSCN/MFgLvAr6dbDgiIpKUqCP8rwJXAQMB21xkZmcBTwOfcvcdlRuY2RpgDUD/tGlnnDhvXp3hioi0Tvcc2Dvy6rTDmGLHU4/82d2PbWTf0IRvZucDe919i5mdXWOznwA/cPdXzOxjwHrg7ZUbufs6YB3AssWLfdPatY3ELCKSuPWnX5Z2CFV9csW0PzW6b5QR/krgAjNbBfQBM83s++7+gdIG7v5C2fbfAr7YaEAiImlq10TfDKEJ392vBq4GKI7wP1Oe7IvL57v7YPHhBRQ+3BURyYxOTvQl9VTpTGFm1wOb3f0O4O/N7AJgDHgRuLw54YmIJGv2+7u58YkPhG/YAepK+O5+L3Bv8edrypZPvgsQqeWuoel87cUBdo91M69nnE/MGWLVwEjaYUmOrT/9Mngi7Shap+ERvkg97hqazvXPz2LUC5XAg2M9XP/8LAAlfUlFHqZwKinhS0t87cWByWRfMupdfO3FASV8aZk8Td9Uo4QvLbF7rLuu5SLNlrfpm2qU8KUl5vWMMzh25MttXs94CtFInuRx6qYWdcuUlvjEnCH6bGLKsj6b4BNzhlKKSDrd7Pd3K9lX0AhfWqI0T68qHWkFTd9Up4QvLbNqYEQJXhKlEX0wJXwRyTwl+mg0hy8imaV5+voo4YtIJuW9pr4RmtIRkczRh7KNUcIXkczQ9E08Svgi0vaU6JtDCV/qoo6X0kpK9M2lhC+RqeOltIo+kE2GEr5Epo6X0gr6QDY5SvgSmTpeSpI0fZM81eFLZLU6W6rjpcShL0+1jkb4Etkn5gxNmcMHdbyUxk3O02v6pmWU8CUydbyUZtGHsulQwu9ASZZOquOlxKUPZdOjhN9hVDop7Urz9OlTwu8wKp2UdqNE3z6U8DuMSielXSjRtx+VZXYYlU5KO1Cyb09K+B1GNwuXNKmmvr1pSqfDqHRS0jCZ5FV909YiJ3wz6wY2A7vc/fyKddOAW4AzgBeA97n7H5sYp9RBpZPSShrRZ0c9I/wrgW3AzCrrPgy85O4nmNklwBeB9zUhPskJtV3OFn1xKpsizeGb2ULgXcC3a2yyGlhf/HkDcK6ZWfzwJA9K3x0YHOvBscnvDtw1ND3t0KQKJfvsijrC/ypwFTBQY/0CYAeAu4+Z2X7g1cCfY0coHU/fHcgG9b7JvtCEb2bnA3vdfYuZnV1rsyrLvMqx1gBrABbOmVNHmNLJ9N2B9qd2CJ0hygh/JXCBma0C+oCZZvZ9dy9/T7cTWATsNLMeYBbwYuWB3H0dsA5g2eLFR/xBkHya1zPO4NiRL0V9dyB9+kC2s4TO4bv71e6+0N0XA5cAv6xI9gB3AKVXxsXFbZTQJRJ9d6D9qJ6+MzVch29m1wOb3f0O4Gbge2b2DIWR/SVNik9yQN8daC+avulcdSV8d78XuLf48zVly0eB9zYzMEnHDXtncutQPxMU3v5dNHCQtXMPJH5efXegPWhU39n0TVuZdMPemfx4qJ/SZ/ATUHxMS5K+pENllvmhhC+Tbi1L9ocZtw71K+F3KE3f5IsSvkyaqHO5ZJembvJJ3TJlUq0Xg14knUPVN/mmEb5Mumjg4JQ5/ALnooGDaYUkTaTpG1HCl0mlefo0qnQkORrRS4kSfkat2TmHh1+ZNvl4xbRXWLfwiC83123t3AOJJXh1xGytTk/0z+y7m817b2J4bA8zeo5j+dwrOGH2eYnvm2Wans2gw8neJv89/Mo01uxs3/5E6ojZWnlI9vcNfp7hsd2AMzy2m/sGP88z++5OdN+sU8LPoMPJvpxNGfG3m6COmNI860+/rOOTPcDmvTcx7qNTlo37KJv33pTovlmnhC8toY6YyctDoi8ZHttT1/Jm7Zt1msOXllBHzGTk9VuyM3qOK07JHLk8yX2zTiP8DFox7RWOvN2AF5e3J3XEbL71p1+Wy2QPsHzuFXRb35Rl3dbH8rlXJLpv1mmEn0HrFr6YWJVOUtQRs3nyNHVTS6mippFKmzj7Zp0SfkZdOGuEHS/2TCbPC2dFT5xB5ZFJlk6qI2Y8eZ2+qeWE2ec1nKTj7JtlSvgZVCpxLFW9lEocgdCEGrQv0PBxJVn6lqw0gxJ+BsW56XdYeaRuJt5eNH0jzaSEn0FxShwb2Velk62n6RtJghJ+BsUpcQzbV6WT6dP0jSRFZZkZFKfEMWhflU6mS62LJWka4WdQnBLHKPuqdLL1NKqXVlDCz6g4JY5B+6p0srU0opdWUsKPIKna9LDjqp1w58paos9rO+FOo4QfIk7Ne5zjJnVeSVfWEj0cbidc6jBZaicMKOlnjD60DZFUW9+w46qdcOfJYrKHfLcT7jQa4YdIqq1v2HHVTrhzZDXRl+S5nXCnUcIPkVRb37Djqp1w9mU90ZfkuZ1wp9GUToikatPDjqua+GzrlGQP+W4n3Gk0wg+RVFvfsOOqnXA2dVKiL8lzO+FOE5rwzawP+DUwrbj9Bne/tmKby4EvA7uKi25y9283N9T0JFWbHnbcR0d62TPWjQN7xrp5dKR3cvsb9s7k1qF+Jii8Tbto4CBr5x4Aki3nVKlodZ3e+yav7YQ7TZQR/ivA29192Mx6gfvN7G53f7Biux+6u97jNckNe2fy46F+Sjcrn4Di44Ja65ZNP5RYOadKRavTt2QlK0ITvrs7MFx82Fv8V3l/PWmyW8sS+mFWXE7NdfePjCfW4jhOW+ZO1InTN9LZIs3hm1k3sAU4Afi6uz9UZbOLzOws4GngU+6+o8px1gBrABbOmdNw0HkwUefy0rokyzlVKlqgRC9ZFalKx93H3f10YCGwwsyWVmzyE2Cxu58K3AOsr3Gcde6+3N2XHzOgLxAFqfWL6QpZV6tssxnlnEkeOwvUzVKyrq6yTHffB9wL/E3F8hfc/ZXiw28BZzQluhy7aOAgR86cORcNHAxcl2Q5Z55LRdeffllHfygr+RClSudY4JC77zOz6cA7gC9WbDPf3QeLDy8AtjU90pwpVdzUqsQJW5dEJU0eS0U7vfpG8iXKHP58YH1xHr8L+JG732lm1wOb3f0O4O/N7AJgDHgRuDypgNMQpxQxqHwyjmXTD3H/yDi7x7o5rmecZdMPxT4mhMfbju2Tew9uoW/obrrG9zHRPZvRgfM41B/tTebPun7Puu7fsJch5jLAmvG38M6Jk1h/+mWFDpH/dhPDY19V7bl0BCsU4bTessWLfdPatamcux6VpYhQmMa45tj9oYmvsrSywHlvhKQftG9l6WV5TEAq8aal9+AWjtq/AfPDf/Dcenl51sWhSf9nXb/nS92/4BUbm1w2zXt484LPAUzpEAmFb5e+df5nlfQlVZ9cMW2Luy9vZF+1VggRp2tleGllY/sGxZRWvGnpG7p7SrIHMD9E39Ddofuu6/7NlGQP8IqNsXnvTeoQKR1JrRVCxClFbKS0Msq+jcSUdLxp6RrfV9fycnup/mFzUBdIdYiULNMIP0ScUsSg8sk4+wbFlFa8aZnonl3X8nJzqf6uZ0bPcTU7QapDpGRZO/+/3BbilCIGlU/G2TcoprTiTcvowHm49U5Z5tbL6EDwPPvs93fzVwv+e80ukOoQKZ1IUzoh4pQiRimtjLNvUEytjjcth/rP4GWIXKUz+cWpJ+CE4puAoC6Q6hApnUQJP4I4pYhr5x5oOGE2WnqZVrxpOdR/RqQyzGrfkg3qAjnvpYfpHx3kYDf0jw4y76WHoQUJ/4HnvsBT+27DmcDo4uTZ72Hl8Vcnfl7pfEr4bSqoMyWgrpV1aKQdwvD267j94EZGewpVS3t64PaDG7lwO8xYcm3I3o174LkvsG3fhsnHzsTkYyV9iUtz+G0qqdLLvGm0982moY2Mdk0tUR3tMjYNbWxGWDU9te+2upaL1EMj/DaVVOllHjSjHcLeGpey1vJm8RpFsLWWi9RDCb9Nhd3EXDc4r65ZNyOZO16Yxqm2PElGV9XkbnozLk2gV1GbSqr0slOtP/2yprYuPmdgNX0TU0tU+yaccwZWN+0c1Zw8+z11LReph0b4bSpKOWieulbWklQ3yxlLruXC7YW5/L3dhZH9OQOrE/3AFg5/MKsqHUmCmqdJZulmJJJHcZqn5WaEH6fFcdC+SbU/jhtzpylvgWwzj+bu1/3PtENiePt1Nd8BPLPv7sS+tBV07KB1r9+1gZW/v4GBkV0MTV/AAyet5ekFF8c+p2RHLhJ+UE17WAIN2vfRkd4p7YQnoPiY2Ek/Tsyd5ogWyAde4h2PfxogcsJqtqA6/d1Hr5jSWnl4bDf3DX4eIHaSfGbf3TWPDdRct+rgQd7x+KfpHS+8dmaO7Ix8DYPOqaSfLbn40DZO3XrQvkm2E1at/WHVWiD3jo+w8vc3pBRRcJ1+kq2Vg44dtG7l72+YTPYlUa+hWkV3jlwk/DgtjoP2TbKdcJyYO8ns93fTNb6/6rqBkV0tjuawoDr9Wi2Um9FaOejYQetqXaso1zDJ5yOtlYuEH6dlcNC+SbYTjhNzpyjdOHxo+oKq62stb4Va9fhzx2u3UG5Ga+WgYweti3MN1Sq6c+Qi4cepWw/aN8l2wnmuta+sqX/gpLUc6p4+ZZtD3dN54KT0qryC6vSTbK0cdOygdXGuoVpFd45cfGgbp8Vx0L6ldUlU6cSJOatqlVmWPlRstMIkCUF1+icUt0miqqV0jHpbOj9dbAXdyDWMck7JBtXhR5DFks4syWo9fVKlij1bP8jGicfY3d3NvPFxVnedytjS70Xad+sfPsrDf9ky+Zpa8aozWHrit2LHJO1DNzFPUKk8cnCsB8cmyyPvGpoea98b9s7kx0P9TGCAMYHx46F+btg7M/kn1UaynOzvG/w8w2O7AZ8sVXxmX/jN04P0bP0g32Urgz09uBmDPT18l630bP1g6L5b//BRHvzLFibMwIwJMx78yxa2/uGjsWKSzqGEHyKLJZ1Z0OzeN62WVKnixonHGO2qeM10dbFx4rHQfR/+yxawiteUWWG5CDmZw48jqZLOWhNpeWiCm+VEX5JUqeLu7hqvmRrLyyVZJiydQQk/RFib4kb33VOjjr9T33Il1eQsLTN6jitO5xy5PI554+MM9lR5zYyHv966qJ7cO/U1JfXTayFEFks6202pnr6TJFWquLrrVPomKl4zExOs7jo1dN8VrzoDKosw3AvLRVDCD7VqYIRrjt3P/J4xDGd+zxjXHLs/cklnrX3Xzj3AewcO0oUDThfOezuwSqcTpm+qOWH2ebx1/meZ0TMPMGb0zOOt8z8bu0pnbOn3uJylzB8bw9yZPzbG5SyNVKWz9MRvcearzqDLHdzpcudMVelImdApHTPrA34NTCtuv8Hdr63YZhpwC3AG8ALwPnf/Y9OjDRFUAhmntLK85r5ej470sqc4Z79nrJtHR3onj7Vs+iHuHxln91g3x/WMs2z61H4xaXTLLO9KOdE9m9GB8zjUH22E2PfirUwbeRBwzLrgtDOBwwk/TrfGRjtTBu0X1wmzz6uZ4O/a/nGeG3l48vHx01ewask3Jh8HXYt9/ScwMVS4bdcE3ezrP4EZZccOer5LT/wWSxt8Po124UxSO8aUZaF1+GZmQL+7D5tZL3A/cKW7P1i2zd8Cp7r7x8zsEuC/uvv7go7b7Dr8yu6SUJg+uebYQh+WWuuSTKCl0sup1TiFkfyy6YcCYwp6PknFfERXSsCtl5dnXRya9PtevJW+kf84YvlvX/MhfnXKl3j9rg1TujVC4Zue95zyldCkP9mZsqxZWd+Ec2H/6iM6U0JhauWt8z/LvJcerrlfkjcyqUz2JaWkH3QtHvnLE4ExV3auhMPPN06yCzoukMg5sxhTO0i0Dt8LhosPe4v/Kv9KrAbWF3/eAJxb/EPRMkElkGl1ngwqvQyLKY2Yq3WlND9E31B4bXnf6ENVl5+64xaAWN0aG+1MGbRfkqol+/LlQdciLOakykEb7cKZpHaMKesiVemYWTewBTgB+Lq7V/7fvQDYAeDuY2a2H3g18OeK46wB1gAsnDMnXuQVGimfTLrzZFCZXFi8aXTL7BrfV9fy2e8vi+VL1Z+teaG6JE63xqDOlB5QHnkwYL80BV2Lvd0Lq64rxZxUOWgjx026W2Y7xpR1kT60dfdxdz8dWAisMLPKacJqo/kj5orcfZ27L3f35ccMNHekGtRdMq3Ok0HdNMNiSiPmie7ZkZdPSfYAVv3ZuhW2i9OtsdHOlEH7pSnoWoTFnFTnyka7cCapHWPKurqqdNx9H3Av8DcVq3YCiwDMrAeYBbzYhPgiCyqBTKvzZFDpZVhMacQ8OnAebr1To7VeRgcOz4nOfn/3kckeih/QTuXAY4suBeJ1vGy0M2XQfkk6fvqKwOVB1yIs5qTKQRvtwpmkdowp66JU6RwLHHL3fWY2HXgH8MWKze6gUI7xH8DFwC+9xV3ZonSXbHXFS6nEMqhBWq2Y0uiWeaj/DF6GqlU6VZN8uXe+t/Df3z0IPsGEdfPYokv51SlfAuJ1vGy4M+Xs82rul6RVS74RWKUTdC1mcHFgzEl1rmy0C2eS2jGmrIsyhz8fWF+cx+8CfuTud5rZ9cBmd78DuBn4npk9Q2Fkf0liEQeIUz6ZlLVzD9SsrQ+LN43nc6j/jCkVOaGJvtzCJfD/tsGBlxjum8/gnOoj3UbMWHIt76Z6ol518CA37HyumDzHeaD/4GQ74De96o184s+3H06sr34jT5ftG1QeGVb2F7RveQlmM58rBJeDxhF03LBzJlUiGScmOVJownf3x4BlVZZfU/bzKPDe5obWPLoheP3qSvQAT26Gf/8RjBWqfCpvkl1ZiljPTbSDBB0XCDxn0L539fcH3rg7zvNJ6lqkRTc5z45cfNNWNwSPruYcfZhf3zWZ7EvKyy7jlGUGCTpu2DmD1oeV/cV5Pkldi7SoRDI7ctE8TTcEj6ahRF9y4KWqi0sliHHKMoM0ctwoMQ3XeG2Uyv7iPJ+krkVadJPz7MjFCF83BK+tNKKPlewBZh5ddXGpBDGpG5EHHTfsnEHrw8r+4jyfdrwpexwqkcyOXCT8PN8QvJamJPlyZ62CnqklneVll0ndiDzouGHnDFofVvYX5/m0403Z41CJZHbkYkonjzcED9LURF/yhkJrjwO/+FXVqpWkbkQe5bi11gXtG3Yj8jjPpx1vyh6HbnKeHbqJeU4kkuQrdFrPe5F2FKd5Wi5G+HnWikRfElaL/bbHr+LUHbdgPo5XfDErqTruoHOGnTcspgee+wJP7bsNZwKji5Nnv4eVx18NxGsFHSbJYychqd+t2iPXTwm/g7Uy2f/0T9u4b/DemrXYb3v8Kk579juTTZfMxznt2e8AcPOicxKp4w46569O+VJg/TgQGNMDz32Bbfs2TG7rTEw+/pCfmFidfdZq+JOq0Vftf2Ny8aFt3jT9A9kIPv/IbwNrsU/dcUuVRtGF5UnVcQedE+K1331q321Vz/nUvtsSrbPPWg1/Gu2cpTaN8DtIq5N8ubBa7FKb5Erm44nVcQedM+j4Udrveo3m185EonX2Wavhb6d2zqIRfkdIY0Rf7sYnPhBai11qk1zJrTuxOu6gcwYdP0r7Xavxv47RlWidfdZq+NNo5yy1KeFnWNqJvlxYLfZjiy6t0ii6sDypOu6gc4bFHBbTybPfU/WcJ89+T6J19lmr4U+jnbPUpimdDGqXJF8urBa7VBlTrWImrOa9UUHnjBJz0LpSNU61Kp1SN84kKmmyVsOfZjtnOZISfoZETvRPbi40MzvwUqHlwVmrJr8YlaSgNsVQSMDlJZHlunf+Cy/bIAAvHxqke+e/QMT/eYPKFIPOCfFa7B531OnsGL6f4bE99PfM5bijTp9c9/SCixNLwkkeOwlptHOW6pTwM6CuEX1Fm2IOvFR4DIkk/dKXreKUC27f+m5+YbugeN/7CeAX7OLcre9mydKfBO6bVJliWNmfygIlizSH3+bqnr6p0qaYsUOF5QmKUy64qSzZTzIrLE/wvEHCyv5UFihZpBF+G0qiTXHN5TGUt1KIUy5YvcCx9vIox49bphhW9qeyQMkijfDbSJJtimsub5I45YK1XoRRXpxJlSmGlf2pLFCySAm/DTS1vLJKm2J6egvLExSnXPAcXwCVTfzcC8sTPG+QsLI/lQVKFmlKJ0VJtilOukqnsjNmnHLBJUt/wrlb380mdjFBYRRyji8I/cA27nmDhJX9qSxQskjtkVPSjrX0kRRLPv3AvrqTa1D5ZFIdINVRsXPl9Xer9sgZktlED1NKPo36SiCDyieBVEorJbv0u22M5vBbpJ3aIDSsSsln1BLIoPLJtEorJbv0u22MRvgJy3ySL1ejtDNKCWQj5ZNJl1ZKdul32xiN8BPSESP6SjVKO6OUQAaVT6ZVWinZpd9tY5Twm6wjE33JWasaLoEMKp9Mq7RSsku/28ZoSqdJOjbJl3vDcu7Z+dcNVdNEKZ9sdWmlZJd+t40JTfhmtgi4BZhH4dvu69z9xoptzgY2AtuLi25z9+ubG2p7ipPof/qnbdy49QF2vzzEvKMGuHLpSt712r+KtnNKHTGDhJVW3tXfz/ULj2d4rHDTk+X9/ZOtkZPqABmno+Lw9uvYNLSRvd0wdxzOGVjNjCXXNjnCI+W13LBe6pZZvygj/DHgH9z9ETMbALaY2c/d/cmK7e5z9/ObH2J7ijui/+mftvG5LfcwOj4GwODLQ3xuyz0A4Um/xR0xS+7+WV/DpZVZK6Mb3n4dtx/cyGhPoanbnh64/eBGLtxOokk/a9dJsiV0Dt/dB939keLPQ8A2oD3vp9YCzZqjv3HrA5PJvmR0fIwbtz4QvnMbdsQMK63MWhndpqGNjHZN7eA52mVsGtqY6Hmzdp0kW+qawzezxcAy4KEqq99sZr8DngM+4+5PVNl/DbAGYOGcOfXGmqpmz9HvfnmoruVTtLAjZrk4pZVZK6PbW+PXXWt5s2TtOkm2RK7SMbMZwK3AJ939QMXqR4DXuvtpwNeA26sdw93Xuftyd19+zMBAozG3VFJVN/OOqv78ay2fIoWOmDc+8YFYpZVZK6ObO17f8mbJ2nWSbImU8M2sl0Ky/1d3v61yvbsfcPfh4s93Ab1mdkxTI01BkpU3Vy5dSV/31DdYfd09XLl0ZfjObdgRM6y0MmtldOcMrKZvYmqfqb4J55yB1YmeN2vXSbIlSpWOATcD29z9KzW2mQfscXc3sxUU/pC80NRIW6gVJZalD2YbqtJpUUfMklJnzDillVkro5ux5Fou3E7Lq3Sydp0kW0K7ZZrZXwP3AY9z+CZE/wS8BsDdv2lmVwAfp1DRMwJ82t1/E3TcduyWmYta+gZUtkIWkfQk2i3T3e8HLGSbm4DMlhGklujbsJY+zNsev4pTd9yC+Thu3Ty26FJ+dcqX0g5LRCLI9TdtUx3Rp1RLH8fbHr+K0579zuRff/NxTnv2OwBK+iIZkLteOqWqm9Snb1Kqpa9X+XTOqTtuOeKtnhWXi0j7y1XCTz3Jl0uplj4O8+o1ibWWi0h7ycWUTlsl+pKZR1dP7gnW0ter8sNat+6qyd2tDa+viByho0f4bTF1U0tKtfRxPLboUiprury4XETaX0eO8Ns2yZdrcS19M5Q+mFWVjkg2dVTCbzTR3/54H1/+5Qye29/F8bMm+Me3D3PhKaPhO8b1huXBCT7Fss1atfeDc1bwuud/zsDILob75jM4Z0VL4klDWLtnkazpiIQfZ0R/++N9XH3nACOHCrNbu/Z3c/WdhX42LUn6tbRh2ebrd20IbIHcSfL0XCU/Mj2H34w5+i//csZksi8ZOdTFl385I9ZxY2vDss2wFsidJE/PVfIjkyP8Zs7RP7e/+t+8Wstbpg3LNhtpj5xVeXqukh+ZG+E3+wPZ42dN1LW8ZVJogVxSa/4+rAVyJ8nTc5X8yEzCT6rE8h/fPsz03qnJfXpv4YPbVLVh2WZYC+ROkqfnKvnR1lM6rSivLH0wm0qVTpCUyjaDOmNGaY/cKfL0XCU/QtsjJyWsPXImauk7UBZbIQ9vv67hvvUqvZSsSbQ9cqsp0Us9hrdfx+0HNzLaU2jrtqcHbj+4kQu3E5r0VXopedM2c/ht3QYhJ7I4ut80tJHRrqk9PEe7jE1DG0P3Veml5E3qI3wleYljb42XT63l5VR6KXmT2gi/e46SvcQ3t0Zn5lrLy6n0UvKmbaZ0JF1ZnM6Bwge0fRNTCw/6JpxzBlaH7qvSS8kbJXzJtBlLruXC/tUcN+aYO8eNORf2R6vSeXrBxdxzylc4MH0hjnFg+kLuOeUr+sBWOlbqc/gicc1Yci3vJloZZqWnF1ysBC+5oRG+ZHY6R0Tqo4QvIpITSvg5p9G9SH4o4YuI5IQSvohITijhi4jkRGjCN7NFZrbJzLaZ2RNmdmWVbczM/tnMnjGzx8zsTcmEK82k+XuRfIlShz8G/IO7P2JmA8AWM/u5uz9Zts15wInFf/8F+EbxvxLHk5tb3g9fRDpX6Ajf3Qfd/ZHiz0PANqCy2chq4BYveBCYbWbzmx5tnjy5Gf79R4fvYXvgpcLjJzenG5eIZFZdc/hmthhYBjxUsWoBsKPs8U6O/KMg9fj1XTB2aOqysUOF5U2g6RyR/Imc8M1sBnAr8El3P1C5usouR9xKy8zWmNlmM9v8/J9Tvh3/NA4AAAbHSURBVGdsuyuN7KMuFxEJESnhm1kvhWT/r+5+W5VNdgKLyh4vBJ6r3Mjd17n7cndffuwxMxqJNz9mHl3f8jpodC+ST1GqdAy4Gdjm7l+psdkdwKXFap0zgf3uPtjEOPPnrFXQ0zt1WU9vYbmISAOiVOmsBD4IPG5mvy0u+yfgNQDu/k3gLmAV8AzwMvCh5oeaM6VqHFXpiEiThCZ8d7+f6nP05ds48HfNCkqK3rBcCV5EmkbftM0Zzd+L5JcSvohITijh54hG9yL5poQvIpITSvgiIjmhhJ8Tms4RESV8EZGcUMIXEckJJXwRkZxQws8Bzd+LCCjhi4jkhhJ+h9PoXkRKlPBFRHJCCV9EJCeU8DuYpnNEpJwSvohITijhi4jkhBJ+h9J0johUUsIXEckJJXwRkZxQwu9Ams4RkWqU8EVEckIJv8NodC8itSjhi4jkhBK+iEhOKOF3EE3niEgQJXwRkZxQwhcRyYnQhG9m/9vM9prZ1hrrzzaz/Wb22+K/a5ofpoTRdI6IhOmJsM13gZuAWwK2uc/dz29KRCIikojQEb67/xp4sQWxSIM0uheRKKKM8KN4s5n9DngO+Iy7P1FtIzNbA6wpPnzF+j9cdZooRccAf047iCpC4vpwywIp047XSjFF044xQXvG1Y4xndTojubu4RuZLQbudPelVdbNBCbcfdjMVgE3uvuJEY652d2X1x9yctoxJmjPuBRTNIopunaMq9Niil2l4+4H3H24+PNdQK+ZHRP3uCIi0lyxE76ZzTMzK/68onjMF+IeV0REmit0Dt/MfgCcDRxjZjuBa4FeAHf/JnAx8HEzGwNGgEs8yjwRrGs06AS1Y0zQnnEppmgUU3TtGFdHxRRpDl9ERLJP37QVEckJJXwRkZxoScI3s24ze9TM7qyybpqZ/dDMnjGzh4oloGnHdLmZPV/WLuIjLYjnj2b2ePF8m6usNzP75+J1eszM3pR0TBHjanlrDTObbWYbzOwpM9tmZm+uWN/yaxUhppZeJzM7qexcvzWzA2b2yYptWnqdIsaUSqsWM/uUmT1hZlvN7Adm1lexvuV5KkJM9ecpd0/8H/Bp4P9QqOWvXPe3wDeLP18C/LANYrocuKkVcZSd84/AMQHrVwF3AwacCTzUJnGdXe0aJhzTeuAjxZ9fBcxO+1pFiKnl16ns3N3AbuC1aV+nCDGl8XpaAGwHphcf/wi4vGKbluapiDHVnacSH+Gb2ULgXcC3a2yymsL/LAAbgHNLZZ4pxtSOVgO3eMGDwGwzm592UK1mhS/6nQXcDODuf3H3fRWbtfRaRYwpTecC/+nuf6pYnuZrqlZMaekBpptZD3AUha4B5VqepyLEVLdWTOl8FbgKmKixfgGwA8Ddx4D9wKtTjgngouLb3A1mtijheAAc+JmZbbFCC4pKk9epaGdxWdpxQbG1hpndbWZvTDie1wHPA98pTsl928z6K7Zp9bWKEhO09jqVuwT4QZXlab2moHZM0OLr5O67gP8FPAsMAvvd/WcVm7U0T0WMCerMU4kmfDM7H9jr7luCNquyLLFa0Ygx/QRY7O6nAvdw+C97kla6+5uA84C/M7OzKta39DqVCYvrEQpvy08DvgbcnnA8PcCbgG+4+zLgIPA/KrZp9bWKElOrrxMAZvYq4ALgx9VWV1mW+GsqJKaWXyczO5rCCH4JcDzQb2aVHQlbnaeixFR3nkp6hL8SuMDM/gj8X+DtZvb9im12AosAim9dZpFsd87QmNz9BXd/pfjwW8AZCcZTOudzxf/uBf4NWFGxyeR1KlpIE97ixY3LW99aYyew090fKj7eQCHZVm7TymsVGlMK16nkPOARd99TZV0qr6mgmFK6Tu8Atrv78+5+CLgNeEvFNq3OU6ExNZKnEk347n61uy9098UU3sL90t0r/0rdAVxW/Pni4jaJ/eWMElPFPOYFwLak4imer9/MBko/A+8EKjuJ3gFcWqysOJPCW7zBtOOyFrfWcPfdwA4zK3UMPBd4smKzll6rKDG1+jqV+W/Unjpp+WsqLKaUrtOzwJlmdlTx3Ody5P/zLc1TUWJqKE8l+UlzxSfKZ1P89B24Hrig+HMfhbd2zwAPA69rg5i+ADwB/A7YBJyccByvK57rd8Xzri0u/xjwseLPBnwd+E/gcWB5C65PlLiuKLtWDwJvaUFcpwObgccovOU/ug2uVVhMaVynoygky1lly9K+TmExtfw6Fc97HfAUhQHN94BpaeepCDHVnafUWkFEJCf0TVsRkZxQwhcRyQklfBGRnFDCFxHJCSV8EZGcUMIXEckJJXwRkZz4/+7LsQ7uIUgUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_boundary(log_reg, axis=[4, 8.5, 1.5, 4.5, 4, -4, 4])\n",
    "plt.scatter(X[y==0, 0], X[y==0, 1])\n",
    "plt.scatter(X[y==1, 0], X[y==1, 1])\n",
    "plt.scatter(X[y==2, 0], X[y==2, 1])\n",
    "plt.show() # 可以看到已经支持了多分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 代码实战OvO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='multinomial', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='newton-cg', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_reg2 = LogisticRegression(multi_class='multinomial', solver='newton-cg')\n",
    "log_reg2.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7894736842105263"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_reg2.score(X_test, y_test) # 正确性明显提高"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:17: UserWarning: The following kwargs were not used by contour: 'linewidth'\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfZBcdZ3v8fd3ZkIyTiYZkIdAAoIFKpJFIqlclty1XKQsCRRwBVZuoStbu8u6u7px3Ye6LBYiJUVZ3nLNypaKeq8oe30gcAElWOoFrkguUAnPOKzLLg8JZAgCSSYhgUz6e//o7slMp/uc7j79Ow99Pq+qFNPnqb/TTL45c86nv23ujoiI9L+BrAsQEZF0qOGLiJSEGr6ISEmo4YuIlIQavohISajhi4iURNsN38wGzexhM/tJk3WXmtnLZvZI7c+f9LZMERFJaqiDbVcD48CCFut/6O6fTF6SiIiE0NYZvpktAc4GvhW2HBERCaXdM/yvAH8PjEZsc4GZvQ/4DfDX7r6pcQMzuwy4DGBk7txTT1i0qMNyRfrXK8NvzboEKYBNTz30W3c/rJt9Yxu+mZ0DbHX3jWb2/hab/Rj4vru/YWafAG4AzmjcyN2vB64HWHbssX73FVd0U7NIX7rhlI9nXYIUwKdXzH2u233buaSzEjjXzJ4FfgCcYWY3ztzA3V9x9zdqD78JnNptQSJlpGYvaYht+O5+ubsvcfdjgYuBu9z9ozO3MbMjZzw8l+rNXRFpw9glg1mXICXRSUpnFjO7Gtjg7rcDf2Vm5wJTwKvApb0pT6S/jV0yyJonPxq/oUgPdNTw3f0e4J7a11fOWH45cHkvC5P+s25ymK++OsrE1CCLhvbxqUMmWTW6O+uyMqNmL2nr+gxfpBPrJoe5+uWF7PHqVcQtU0Nc/fJCgNI2fTV7SZtGK0gqvvrq6HSzr9vjA3z11aikb//STVrJghq+pGJiqvmNyVbLRaT31PAlFYuG9nW0vJ/p7F6yooYvqfjUIZPMs8qsZfOswqcOmcyoomyo2UuWdNNWUlG/MVvmlM7YJYPwZNZVSJmp4UtqVo3uLlWDn0kRTMkDXdIRSYGaveSBGr5IYLpuL3mhhi8SkObkSJ6o4YsEpEs5kidq+CKB6FKO5I0avkgAavaSR2r40pF1k8Oc9dzhLPv3IznrucNZNzmcdUm5o+v2klfK4UvbNPGyPbpuL3mlM3xpmyZextOlHMkzNXxpmyZeRlOzl7xTw5e2aeJla7puL0Wghi9t08TL1nTdXopAN22lbZp42Zwu5UhRqOH3oZAfFl7miZciRaeG32cUnUyXzu6lSHQNv88oOpkeNXspGjX8PqPoZDrU7KWI1PD7jKKT4anZS1Gp4fcZRSdFpBXdtO0zik6GpbN7KbK2G76ZDQIbgBfc/ZyGdXOB7wKnAq8AH3H3Z3tYp3RA0UkRaaaTSzqrgfEW6/4YeM3djwf+Efhi0sKkXIowdlln91J0bTV8M1sCnA18q8Um5wE31L5eC3zAzCx5eVIG9fcObJkawrHp9w7kqemr2Us/aPcM/yvA3wOVFusXA5sA3H0K2A68NXF1Ugp5f++Amr30i9iGb2bnAFvdfWPUZk2WeZNjXWZmG8xsw28nlRqRqjy/d0DNXvpJO2f4K4FzzexZ4AfAGWZ2Y8M2m4GjAcxsCFgIvNp4IHe/3t2Xu/vyQ0fzcfYm2dN7B0TSEdvw3f1yd1/i7scCFwN3uXvjLNjbgfqp0IW1bQ44wxdpJq/vHdDZvfSbrnP4ZnY1sMHdbwe+DXzPzJ6memZ/cY/qkxLI43sH1OylH3XU8N39HuCe2tdXzli+B7iol4VJNq7ZuoCbJ0eoUP3174LRXVxx+I7gz6v3DoiEp3fayrRrti7gpskR6vfgK1B7TCpNPy90di/9SrN0ZNrNM5r9flZbXg5q9tLP1PBlWqs3WbRa3m/0QeTS79TwZVqrH4ay/JDog8il35Xl77K04YLRXRz4fjmvLe9vupQjZaCbtjKtfmM2i5ROltTspSzU8Avqss2H8OAbc6cfr5j7BtcvOeDNzR274vAdwRr8usnhXGXtpdie3nYnG7Zex86pl5g/dATLD/8kx4+dFXzfItMlnQLa3+xt+s+Db8zlss2HZFxZa3mdiKmz+2J6etud3LvlC+ycmgCcnVMT3LvlCzy97c6g+xadGn4B7W/2M9msM/68yeNETDX74tqw9Tr2+Z5Zy/b5HjZsvS7ovkWnhi+pyNtETEUwi23n1EsdLe/VvkWnhi+pyNNEzLFLBhXBLLj5Q0d0tLxX+xadGn4BrZj7Bs3ik9Xl+ZSniZhq9sW3/PBPMmjzZi0btHksP/yTQfctOqV0Cuj6Ja8GS+mEkpeJmLpu3x/qiZpukjZJ9i06NfyCOn/hbja9OjTdPM9f2H7jjIpHhoxOZj0Rc+ySQXgys6eXHjt+7Kyum3SSfYtMDb+A6hHHeuqlHnEEYhtq1L5A18ctAl3KkbLTNfwCShJxjNo3j9HJXtGlHBGd4RdSkohjN/vm4cPEk1CzF6nSGX4BJYk4Ru2bp+hkryhvL7KfGn4BJYk4Ru2bp+hkr+i6vch+uqRTQEkiju3sm3V0sld0KUdkNjX8gkoScYzaN+voZK+o2YscSA2/DaGy6XHH1ThhyYuyjhPuN2r4MZJk3pMcN9TzloHO7nurPk64PmGyPk4YUNMvGN20jREqmx533H7OxIekZt97ZR4n3G/U8GOEGusbd9y8jRMuAkUwwyjzOOF+o4YfI1Q2Pe64/ZiJD0kjj8Mp8zjhfqOGHyNUNj3uuP2YiQ9JzT6cMo8T7je6aRsj1FjfuOPmZZxwEei6fVhlHifcb2IbvpnNA34JzK1tv9bdP9ewzaXAl4AXaouuc/dv9bbU7ITKpscd9+Hdc3hpahAHXpoa5OHdc6a3v2brAm6eHKFC9de0C0Z3ccXhO4Cwcc68RUXV7NNR1nHC/aadM/w3gDPcfaeZzQF+ZWZ3uvv9Ddv90N31O16PXLN1ATdNjlD/sPIK1B5XtVq3bHhvsDinoqIixRZ7Dd+rdtYezqn9afx8Pemxm2c09P2MmydHIteFjHPmLSqqs3uRzrR109bMBs3sEWAr8HN3f6DJZheY2WNmttbMjm5xnMvMbIOZbfjtpG4+RqlELI9aFzLOqaioSLG11fDdfZ+7nwIsAVaY2dKGTX4MHOvuJwO/AG5ocZzr3X25uy8/dFRvIIrS6n/MQMy6kHHOPEVFdXYv0rmOYpnuvg24B/hQw/JX3P2N2sNvAqf2pLoSu2B0FwdeOXMuGN0VuS5knDMvUVE1e5HutJPSOQzY6+7bzGwYOBP4YsM2R7r7ltrDc4HxnldaMvXETaskTty6EEmaPERF1exFutdOSudI4AYzG6TaW37k7j8xs6uBDe5+O/BXZnYuMAW8ClwaquAsJIkiRsUnk1g2vJdf7d7HxNQgRwztY9nw3sTHhPh6sxyf3KrZv+OFtaz812sY3f0Ck8OLue+dV/CbxRe2dcyoKZCaECn9Jrbhu/tjwLImy6+c8fXlwOW9LS0fkkQRo6KVcU0/at+o6CWQSb1ZeccLaznz8c8wZ1/1e1uwezNnPv4ZgNimHzUFEtCESOk7Gq0QI0kUMSo+mWTfqJqyqje0Vmf3K//1mulmXzdn325W/us1sceMmgKpCZHSjzRaIUaSKGJUfDLJvt3UFLrekMYuGYQnm68b3f1CR8tn6mYKpCZESpHpDD9GkihiVHwyyb5RNWVVbwhjlwzGjjyeHF7c0fKZoqZAakKk9CM1/BhJoohR8ckk+0bVlFW9vdbY6FefdGPT7e575xXsHRyetWzv4DD3vfOK2OeImgKpCZHSj3RJJ0aSKGI70cok+0bVlHa9vdTJB5nUb8x2k9JpZwqkUjrST9Tw25AkinjF4Tu6bpjdRi+zqrcXopr96pNubDr3/jeLL2w7htkoagrkotceZGTPFnYNwsieLSx67UFIoeHf9+K1PLXtFpwKxgDvGvswK4/qyxCcpEwNP6ei4qDQffQyr9o9q2/V9Htt5zOf59Zdt7FnqJpaemkIbt11G+c/A/OP+1zM3t2778VrGd+2dvqxU5l+rKYvSekafk6Fil7mUR4/i/buydvYMzA7orpnwLh78ragz/vUtls6Wi7SCTX8nIqKXvbT1Mpumn2rG7i9tLVFWa2W94q3CMG2Wi7SCTX8nAoVvcyLdiKXUUI3/cNbvJStlveKtfgr2Wq5SCf0U5RToaKXeZDHSziNfn/0POZVZkdU51Wc3x89L+jzvmvswx0tF+mEGn5OrRrdzZWHbefIoSkM58ihKa48bPt0AqfVujxLelbfKORZ/vzjPsf5I+dxxJRj7hwx5Zw/cl7QG7ZQvTF74tiF02f0xgAnjl2oG7bSE+aezacVLjv2WL/7ivg3x0h/CHlWn0ZqRyQvPr1i7kZ3X97NvqWJZSYZcRy1b6jxx0lrzoueNfpfb4BfroMdr8GCg+F9q+DdXf3M98zOZz7P3ZO3sXWwem3/90f3/wYQcrRytyOdQ42RluIoRcNPMuI4at+Hd88JNk44Sc150dNm/9MfwVTtjWc7Xqs+Bnj38tSy+TNF5fQnDl4RbLRytyOdV+3aFWSMtJp+sZTiGn6S3HrUviHHCRc9a9/TSzi/XLe/2ddN7a0uz0hUTj/kaOVuRzqHGiMtxVKKM/wkufWofVvd/ehFYrrIWfueX6/f8Vpny1MQldP3LsYut6vbkc5pj5GWfCrFGX6S3HrUviHHCRcxa9/rFM60BQfHLk/jzVgzReX0Q45W7nakc6gx0lIspWj4SXLrUfuGHCdctKx90Gz9+1bB0JzZy4bmVJfPkGbTj8rphxyt3O1I51BjpKVYSnFJJ8mI46h96+tCpHSS1Jy24G+kqqdx2kjppHUDd/5xn+P8Z2ia0jm+tk2IVEu3I51/M1ZdF2qMtBRDKRo+JBsZHCVunHCSSGeomnsl1XfMvnt55jHMRhMHr2DX3gfxqZfYNe8IJg5eMd3so8Yuxxl64mPcVnmMicFBFu3bx3kDJzO19HvT66OOveflW3h97xYAXt+7hT0v3zI90jnUGGkpjlJc0kmiHo/cMjWEY9PxyHWTw4n2vWbrAm6aHKGCAUYF46bJEa7ZuiD8N9UDeR6PkMalnXpUcefUBODTUcWnt92Z6LhDT3yM7/AEW4aGcDO2DA3xHZ5g6ImPxe77xL/9Kfe/uZGKGZhRMeP+NzfyxL/9aaKapH+o4ccoYqQzpGA3ZgsmVFTxtspj7Blo+JkZGOC2ymOx+z745kawhp8ps+pyEdTwY4WKdLaKbuZ5CG6RGn3os/xQUcWJwRY/My2Wz1TEnylJlxp+jCJGOkMoUrOvCzpcLVBUcdG+Fj8zLZbPVLSfKUmffhZiFDHS2Uu6hNNcqKjieQMnM6/S8DNTqXDewMmx+6446FRoHIboXl0ughp+rCSjiKP2veLwHVw0uosBHHAGcC7q4eC1XuiHRh/qLP/4sbP4vSM/y/yhRYAxf2gRv3fkZxMnWaaWfo9LWcqRU1OYO0dOTXEpS2eldFpZesI3Oe2gUxlwB3cG3DntoFNZesI3E9Uk/SM2lmlm84BfAnNr26919881bDMX+C5wKvAK8BF3f7bn1caIikAmmTyZJB758O45vFQbw/DS1CAP754zfaxlw3v51e59TEwNcsTQPpYNz54Xk8W0zDm7NjJv8k4G922Dr3c4lfJnN8Gj94NXwAbgPafBBy/avz7BxMs71q9jzfPjTAwYiyrO6mNO5OzTq2+8uuO5cdY8cR8Tr0+y6C2jrF66krPfduL0fj9+7stNJ1omFRVVXPfMn/Pi7genHx81vIJVx31t+nHU5MptI8dTmXwSgAqDbBs5nvkzjh01uXLpCd9kaZffT7dTOEPKY01FFjsP38wMGHH3nWY2B/gVsNrd75+xzV8AJ7v7J8zsYuC/uPtHoo7b63n4jdMloXr55MrDtgO0XBeygdajl7PTONUz+WXDeyNrivp+QtU8Z9dGRnaunT2obGgOfOgP4hvzz26CR9YfuPyU06tNv3HiZQfHvmP9Oq7aND4rvTKvUuGqo0+Excdx1cZfsGff1P51g0NcdeqZ8MIzTfYL/0Emjc2+rt703/HC2lmTK6H6rtdf/M6XeejNJ6tTOGcMZptZc+PkSqheSkr620XUcYEgz1nEmvIgyTz82Es6XrWz9nBO7U/jvxLnATfUvl4LfKD2D0VqoiKQWU2ejIpextWURc0jlZ92P5Xy0fujlyeYeLnm+fGmUcU1z1fP7Gc2e4A9+6ZY88R9LfarTrQMqVmzn7k8anJl1BROCBcH7XYKZ0h5rKno2rqGb2aDZvYIsBX4ubs/0LDJYmATgLtPAduBtzY5zmVmtsHMNvx2srczYaIikFlNnoyKycXVlHbNY5cMJptK6S2+2/ryBMeeGGh+7jAxYEy83vznaOL1yZb7tZp0mZaoyZVRUzghXBw06rhZTcvMY01F11bDd/d97n4KsARYYWaNlwmb/c064FqRu1/v7svdffmho709U42KQGY1eTIqJhdXU1o1z0rhtDGVsiVr8d3Wlyc49qJK88uOiyrOorc0/zla9JbRlvu1mnSZlqjJlVFTOCFcHLTbKZwh5bGmousopePu24B7gA81rNoMHA1gZkPAQuDVHtTXtqgIZFaTJ6Oil3E1pVHzASmcNqdSNvWe06KXJzj26mNObBpVXH3MiaxeupJ5g7OzB/MGh1i9dGWL/aoTLUM6anhF5PKoyZVRUzghXBy02ymcIeWxpqJrJ6VzGLDX3beZ2TBwJvDFhs1uBz4O/D/gQuAuT/nT0duZLpl24qUesYwakNaqppDTMlvGLTuYSnmAehqnVUonwbHPPn0VrKdlSgdontJ524lN93t6YbgbtgCrjvtaZEqnnsZpltKZz4Utp3BCuMmV3U7hDCmPNRVdOymdk6nekB2k2rN+5O5Xm9nVwAZ3v70W3fwesIzqmf3F7v4fUcftdUonTj98IHgvBM3Wx8UuQ30QedRxm6xb41+Z3jUqHhkX+0vyoeBJ9s0jRSTTkySlE9vwQ0mz4WcRccyj4M0+KnaZIJbZ9fNCy3Vr/CuR8ch1IyORsb+ofeMad5J98yhUVFSaCxrL7AdF/0DwpFIZjxAXuwz1QeRRx41Yt/qkGyPjkXGxvyQfCp5k3zxSRLI4SvEBKEX+QPCkUhuPEBe7DPVB5N0ct7Zuwe7NTVeP7n6BnS1+NuqxvyQfCp5k3zxSRLI4SnGGX8QPBO+FVGfhxMUuk0Q+u33eLmuaHF4cG/tL8qHgSfbNI0Uki6MUDb9oHwieVCYTLuNil0kin90+bxs1tYpHxsX+knwoeJJ980gRyeIoxSWdIn0geFKZTbiMi10miXwmed6YmuYAO/7P/z0gLRP3QeRR0co4SfbNI33IeXGUIqVTFv0wzjgra578aNYliLQlSUqnFGf4/S4vjT5qTDEQOT45dt9uxYxs3v+8/8jI0KKOxu/e9+K1PLXtFpwKxgDvGvswK4+6HAibsy9ahj9URl/Z/86p4Rdcnpr9zDHFW16f5KqNvwCoNu7G8clemX58xzuXRu/brYjn5IMXHVDzzqkJ7t3yhenNZ2bLZ647fuws7nvxWsa3rd1/aCrTj//IT5iVs1+wezNnPv4ZgMSNuTHD38tjh9CY0W98HfN23H5Xipu2/ShvHz0YNaYYiByfHLtvt2JGNjd73nbH7z617Zamh35q2y1Bc/ZFy/BnMc5ZWtMZfgHlqdHXRY0pBiLHJ8fu262Ykc2tjh+VH6+v8xbDr51K0Jx90TL8WYxzltZ0hl8weWz2QOSYYiByfHLsvt2KGdnc6vjtjN+1Fn91jIGgOfuiZfizGOcsranhF0TeLuE0ihpTDESOT47dt1sxI5tbPe9n33tKbLb8XWMfbnrod419OGjOvmgZ/izGOUtruqRTAHlu9HX1m6stkzYR45PPrh2j5ymdmJHNUTU/vTM6W15P4zRL6fym9vQhkjRFy/BnOc5ZDqQcfs511exDjSEO+LxfuPUb3PTmrunPDbjooBE+e/6fBX/eKJ+6762RDUWxQMmCcvh9qOuz+sZxwTteqz6GsE0/wfN+4dZv8MM3d0Htc+8rUH186zfim36g7/eO58a5d8s9LWN/igVKEekafg4luoQTagxxwOe9aUazn2ZWXR7weaOseeK+yNifYoFSRGr4OZP4en2oMcQBn7dFeLLl8l49b5S4yKZigVJEavg50bMUTqgxxAGft9UPYVs/nIG+36jI5sz/tlovkkdq+DnQ0xROqDHEAZ/3ooNGoDE84F5dHvB5ozSLbM6M/SkWKEWkm7YZChK3DDWGOODzfvb8P4NuUzqBvt/GyGbjYDXFAqWIFMvMSBGy9U0liUBG7RsoWtnLKZwaoZwvZY3FKpZZIIVt9JAsAhm1LwSLVvZyCufqk25U088JxWK7o2v4KSp0s4dkEciofQNGK4NM4ZTMKRbbHTX8lBS+2UOyCGTUvilHK5NM4Vx90o1d7yu9o1hsd9TwA8v70LOOJIlARu2bcrQy6RRONf3sKRbbHTX8gPqm0dcliUBG7ZtitLInUzglc4rFdkc3bQPpu2YPySKQ7ewbOFrZy8/K1Q3cbCkW253Yhm9mRwPfBRZRfbf79e6+pmGb9wO3Ac/UFt3i7lf3ttTiaLfZJ4oMZjURM0FNd4yMsOboo5h4fbT6/Y6MTI9G5t3Lg9R/9ttO7LrB37F+HWueH2diwFhUcVYfcyJnnx74DWyUN27YqePHztLr0qF2zvCngL9x94fMbBTYaGY/d/dfN2x3r7uf0/sSi6OTs/pEkcE8TsSEyJp6HZEM7Y7167hq0zh7BqtXPbcMGldtGof1TDf9EGf5ihtKSLHX8N19i7s/VPt6EhgH8vl5ahnq9BJOoshgHidixtRUtIjkmufH2TMw+6/HnoEB1jw/PmtZr2/gKm4oIXV0Dd/MjgWWAQ80Wf27ZvYo8CLwt+7+ZJP9LwMuA1hyyCGd1ppL3V6rTxQZLNJEzNq6YB9UHsjEgHW0vFcUN5SQ2k7pmNl84Gbg0+6+o2H1Q8Db3P09wFeBW5sdw92vd/fl7r780NGEH1CdA0luzCaKDOZxImZMTcE+qDyQRZXmI0eaLe/lWb7ihhJSWw3fzOZQbfb/4u63NK539x3uvrP29Tpgjpkd2tNKc6QX2fpEkcE8TsSMqaloEcnVx5zIvMrsifzzKhVWHxP2foPihhJSOykdA74NjLv7l1tsswh4yd3dzFZQ/YfklZ5WmhO9ilsmigzmeSJmi3UhI5IhnH36KlhP2ymdXt3AVdxQQoqdlmlm/xm4F3ic/R9C9A/AMQDu/nUz+yTw51QTPbuBz7j7+qjjFnFaZl9m66VnlMuXNASdlunuvwIi71S5+3VA38YIgjX6PGbp4/zsJnj0fvAK2AC85zT44EVZV5ULejOW5J1GK8QI2ux/+qP9CZd6bv3XG8I8Xy/87CZ4ZH212UP1v4+sry4XQHN2JN/U8CMEvYSTVZY+iUfv72y5iOSKGn4Lwa/XZ5WlT8IrnS0vKZ3lS15peFqD1G7MLji4eXMPnaVPwgaaN3fTeYNIEehv6gyppnCyytIn8Z7TOlteYjrLlzxSwyejDyl593L40B/sP6NfcHD1cZ5TOh+8CE45ff8ZvQ1UHyul05SavuRN6S/pjF0yyK2Pz+NLd83nxe0DHLWwwt+dsZPzf2dP/M5JxY0EzmNsc8lx8B/j1ZpGF1Yf96s8vv4iCZT2DL9+Vn/r4/O4/CejvLB9EMd4Yfsgl/9klFsfnxd/kJDyGNvMY02h9Oh71Vm+5EkpG/7Myzdfums+u/fOfhl27x3gS3fNT7us2fIY28xjTaH08HtV05e8KF3Db7xW/+L25i9Bq+WpyWNsM481hVKm71VKozQNv9WN2aMWNs+Qt1qemqxGIEfJY02h9Ph71Vm+5EEpGn5UAufvztjJ8JzZzX14TvXGbabyGNvMY02hlOl7ldLo+4YfF7c8/3f2cO05kyxeuA/DWbxwH9eeM5lOSidKHmObeawplADfq87yJWux45FDSWM8ssYZl8Md69e1Pbf+ABlELzVRU5IIOh65iNToy+OO9eu4atM4ewarv6xuGTSu2jQO64lv+vXoZT2NU49eQn/+1iKl13eXdNTsy2XN8+PsGZj9Y7xnYIA1z4/H75xRzFSXdiQrfXOGr0ZfThMDzT+bp9XyWRS9lJLpizN8NfvyWlRpfg+q1fJZMoyZ6ixfslD4hq9mX26rjzmReZXZsdp5lQqrj2njw9Ezjl6q6UvaCntJR41eoHZjdj3dpXTqN2Y1IE1KopCxTDV76SeKaUonShPLVKMXEeleYa7hq9lLv9K1fElLIRq+mr30OzV9SUPuG76avYhIb+S24WfyObMiGdJZvoSWy4avRi8i0nuxDd/Mjjazu81s3MyeNLPVTbYxM/snM3vazB4zs/d2W5CavZSZzvIlpHZimVPA37j7Q2Y2Cmw0s5+7+69nbHMWcELtz38Cvlb7b9vU6JvIYHSvZG/1STcqmy9BxJ7hu/sWd3+o9vUkMA4sbtjsPOC7XnU/MGZmR7ZbhJp9E/XRvfVBXvXRvb/ekG1dIlJYHV3DN7NjgWXAAw2rFgObZjzezIH/KBxAN2YjZDS6V/JBl3YkhLYbvpnNB24GPu3uOxpXN9nlgJkNZnaZmW0wsw2vVjL+zNi80+heEemxthq+mc2h2uz/xd1vabLJZuDoGY+XAC82buTu17v7cndfftih87uptzwyHN0r+aCzfOm1dlI6BnwbGHf3L7fY7HbgD2tpndOA7e6+pYd1lk/Go3slH9T0pZfaSemsBD4GPG5mj9SW/QNwDIC7fx1YB6wCngZeB/6o96WWjEb3ikiPxTZ8d/8Vza/Rz9zGgb/sVVFS8+7lavCimKb0TC7faSsis+nSjvSCGr6ISEmo4YsUhM7yJSk1fBGRklDDFykQneVLEmr4IgWjpi/dUsMXESkJNXyRAtJZvnRDDV9EpCTU8EUKSmf50ik1fJECU9OXTqjhi4iUhBq+SMHpLF/apYYvIlISavgifUBn+VDjIcUAAAXJSURBVNIONXwRkZJQwxfpEzrLlzhq+CJ9RE1foqjhi4iUhBq+SJ/RWb60ooYv0ofU9KUZNXwRkZJQwxfpUzrLl0Zq+CJ9TE1fZlLDFxEpCTV8kT6ns3ypU8MXESkJNXyREtBZvkAbDd/M/oeZbTWzJ1qsf7+ZbTezR2p/rux9mSKSlJq+DLWxzXeA64DvRmxzr7uf05OKREQkiNgzfHf/JfBqCrWISGA6yy+3ds7w2/G7ZvYo8CLwt+7+ZLONzOwy4LLawzds5I+bXibK0KHAb7Muook81qWa2pPDmv44hzUBuXytclnTO7vd0dw9fiOzY4GfuPvSJusWABV332lmq4A17n5CG8fc4O7LOy85nDzWBPmsSzW1RzW1L4919VtNiVM67r7D3XfWvl4HzDGzQ5MeV0REeitxwzezRWZmta9X1I75StLjiohIb8Vewzez7wPvBw41s83A54A5AO7+deBC4M/NbArYDVzs7Vwnguu7LTqgPNYE+axLNbVHNbUvj3X1VU1tXcMXEZHi0zttRURKQg1fRKQkUmn4ZjZoZg+b2U+arJtrZj80s6fN7IFaBDTrmi41s5dnjIv4kxTqedbMHq8934Ym683M/qn2Oj1mZu8NXVObdaU+WsPMxsxsrZk9ZWbjZva7DetTf63aqCnV18nM3jnjuR4xsx1m9umGbVJ9ndqsKZNRLWb212b2pJk9YWbfN7N5DetT71Nt1NR5n3L34H+AzwD/i2qWv3HdXwBfr319MfDDHNR0KXBdGnXMeM5ngUMj1q8C7gQMOA14ICd1vb/Zaxi4phuAP6l9fRAwlvVr1UZNqb9OM557EJgA3pb169RGTVn8PC0GngGGa49/BFzasE2qfarNmjruU8HP8M1sCXA28K0Wm5xH9S8LwFrgA/WYZ4Y15dF5wHe96n5gzMyOzLqotFn1jX7vA74N4O5vuvu2hs1Sfa3arClLHwD+3d2fa1ie5c9Uq5qyMgQMm9kQ8BaqUwNmSr1PtVFTx9K4pPMV4O+BSov1i4FNAO4+BWwH3ppxTQAX1H7NXWtmRweuB8CBn5nZRquOoGg0/TrVbK4ty7ouqI3WMLM7zeykwPW8HXgZ+J+1S3LfMrORhm3Sfq3aqQnSfZ1muhj4fpPlWf1MQeuaIOXXyd1fAP478DywBdju7j9r2CzVPtVmTdBhnwra8M3sHGCru2+M2qzJsmBZ0TZr+jFwrLufDPyC/f+yh7TS3d8LnAX8pZm9r2F9qq/TDHF1PUT11/L3AF8Fbg1czxDwXuBr7r4M2AX8t4Zt0n6t2qkp7dcJADM7CDgXuKnZ6ibLgv9MxdSU+utkZgdTPYM/DjgKGDGzjzZu1mTXkH2qnZo67lOhz/BXAuea2bPAD4AzzKxxXN9m4GiA2q8uCwk7nTO2Jnd/xd3fqD38JnBqwHrqz/li7b9bgf8NrGjYZPp1qllCD37FS1qXpz9aYzOw2d0fqD1eS7XZNm6T5msVW1MGr1PdWcBD7v5Sk3WZ/ExF1ZTR63Qm8Iy7v+zue4FbgNMbtkm7T8XW1E2fCtrw3f1yd1/i7sdS/RXuLndv/FfqduDjta8vrG0T7F/OdmpquI55LjAeqp7a842Y2Wj9a+CDQOMk0duBP6wlK06j+ivelqzrspRHa7j7BLDJzOoTAz8A/Lphs1Rfq3ZqSvt1muG/0vrSSeo/U3E1ZfQ6PQ+cZmZvqT33Bzjw73yqfaqdmrrqUyHvNDfcUX4/tbvvwNXAubWv51H91e5p4EHg7Tmo6VrgSeBR4G7gXYHreHvtuR6tPe8VteWfAD5R+9qAfwb+HXgcWJ7C69NOXZ+c8VrdD5yeQl2nABuAx6j+yn9wDl6ruJqyeJ3eQrVZLpyxLOvXKa6m1F+n2vN+HniK6gnN94C5WfepNmrquE9ptIKISEnonbYiIiWhhi8iUhJq+CIiJaGGLyJSEmr4IiIloYYvIlISavgiIiXx/wFT3ZM5uvxUWAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_boundary(log_reg2, axis=[4, 8.5, 1.5, 4.5, 4, -4, 4])\n",
    "plt.scatter(X[y==0, 0], X[y==0, 1])\n",
    "plt.scatter(X[y==1, 0], X[y==1, 1])\n",
    "plt.scatter(X[y==2, 0], X[y==2, 1])\n",
    "plt.show() # 可以看到已经支持了多分类，而且OvO的效果明显比上面的OvR好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用所有的数据，不再只取前两个特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = datasets.load_iris()\n",
    "X = iris.data # 取全部数据\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=666)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.使用默认地方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:469: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.\n",
      "  \"this warning.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9473684210526315"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_reg = LogisticRegression() # 默认方法，不支持多分类\n",
    "log_reg.fit(X_train, y_train)\n",
    "log_reg.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.使用OvO方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_reg_ovo = LogisticRegression(multi_class='multinomial', solver='newton-cg') # OvO\n",
    "log_reg_ovo.fit(X_train, y_train)\n",
    "log_reg_ovo.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.使用OvR方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9473684210526315"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_reg_ovr = LogisticRegression(multi_class='ovr') # OvR\n",
    "log_reg_ovr.fit(X_train, y_train)\n",
    "log_reg_ovr.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 可以看到OvO解决多分类的准确率最高"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 上面是逻辑回归中的OvR和OvO，实际sklearn有单独的类取支持OvO和OvR\n",
    "> 只要把符合sklearn的机器学习算法对象传入这两个类就可以实现多分类任务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9473684210526315"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.multiclass import OneVsRestClassifier\n",
    "ovr = OneVsRestClassifier(log_reg) # 对默认的对象改成成OvR方式\n",
    "ovr.fit(X_train, y_train)\n",
    "ovr.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.multiclass import OneVsOneClassifier\n",
    "ovo = OneVsOneClassifier(log_reg)\n",
    "ovo.fit(X_train, y_train)\n",
    "ovo.score(X_test, y_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
